---
title: MLOps reporting for unstructured models
description: Report MLOps statistics from custom inference models created with an unstructured regression, binary, or multiclass target type.
section_name: MLOps
maturity: public-preview
---

# MLOps reporting for unstructured models

!!! info "Availability information"
    MLOps Reporting from Unstructured Models is off by default. Contact your DataRobot representative or administrator for information on enabling this feature.

    <b>Feature flag:</b> Enable MLOps Reporting from Unstructured Models

Now available for public preview, you can report MLOps statistics from Python custom inference models [created in the Custom Model Workshop](custom-inf-model) with an **Unstructured (Regression)**, **Unstructured (Binary)**, or **Unstructured (Multiclass)** target type:

![](images/unstructured-model-reporting.png)

With this feature enabled, when you [assemble an unstructured custom inference model](unstructured-custom-models) in Python, you can read the `mlops` input argument from the `kwargs` as follows:

``` python
mlops = kwargs.get('mlops')
```

For an example of an unstructured Python custom model with MLOps reporting, see the [DataRobot User Models repository](https://github.com/datarobot/datarobot-user-models/tree/master/model_templates/python3_unstructured_with_mlops_reporting){ target=_blank }.

**************************************************

## Unstructured custom model reporting methods {: #unstructured-custom-model-reporting-methods }

If the value of `mlops` is not `None`, you can access and use the following methods:

### `report_deployment_stats` {: #report-deployment-stats }

Reports the number of predictions and execution time to DataRobot MLOps.

``` python
report_deployment_stats(num_predictions: int, execution_time: float)
```

Argument          | Description
------------------|------------------
`num_predictions` | The number of predictions.
`execution_time`  | The time, in milliseconds, that it took to calculate all predictions.

**************************************************

### `report_predictions_data` {: #report-predictions-data }

Reports the features, along with their predictions and association IDs, to DataRobot MLOps.

``` python
report_predictions_data(features_df: pandas.DataFrame, predictions: list, association_ids: list, class_names: list)
```

Argument      | Description
--------------|------------------
`features_df` | A dataframe containing all features to track and monitor. Exclude any features you don't want to report from the dataframe.
`predictions` | A list of predictions. <ul><li>For _regression_ deployments, this is a 1-dimensional list containing prediction values (e.g., `[1, 2, 4, 3, 2]`).</li><li>For _classification_ deployments, this is a 2-dimensional list, where the inner list is the probabilities for each class type (e.g., `[[0.2, 0.8], [0.3, 0.7]]`).</li></ul>
`association_ids` | (Optional) A list of association IDs corresponding to each prediction. Association IDs are used to calculate accuracy and must be unique for each reported prediction. The number of `predictions` should equal the number of `association_ids` in the list.
`class_names` | (Classification only) A list of the names of predicted classes (e.g., `["class1", "class2", "class3"]`). For classification deployments, the class names must be in the same order as the prediction probabilities reported. If the order isn't specified, the prediction order defaults to the order of the class names on the deployment. This argument is ignored for regression deployments.

**************************************************

## Local testing {: #local-testing }

To test an unstructured custom model with MLOps reporting locally, you must use the [`drum` utility](custom-model-drum) with the following input arguments (or the corresponding environment variables):

Input argument | Environment variable  | Description
---------------|-----------------------|-------------
`--target-type`      | `TARGET_TYPE`   | Must be `unstructured`.
`--webserver`        | `EXTERNAL_WEB_SERVER_URL` | The DataRobot external web server URL.
`--api-token`        | `API_TOKEN`     | The DataRobot API token.
`--monitor-embedded` | `MLOPS_REPORTING_FROM_UNSTRUCTURED_MODELS` | Enables a model to use MLOps library to report statistics.
`--deployment-id`    | `DEPLOYMENT_ID` | The deployment ID for monitoring model predictions.
`--model-id`         | `MODEL_ID`      | The deployed model ID for monitoring predictions.